Systems and methods for hyperspectral medical imaging using real-time projection of spectral information

ABSTRACT

Under one aspect, a method of displaying medical information about a subject having a plurality of regions includes: resolving light obtained from each region of the plurality of regions into a corresponding spectrum; selecting a portion of each spectrum, the selected portion including medical information about the corresponding region; constructing an image based on the selected portion of each spectrum; and projecting the image onto the subject. Under another aspect, a method of displaying medical information about a subject that has a plurality of regions includes: resolving light obtained from each region of the plurality of regions into a corresponding spectrum; selecting a portion of each spectrum, the selected portion including medical information about the corresponding region; constructing a spectral image based on the selected portion of each spectrum; displaying an image of the plurality of regions; and displaying the spectral image overlying the image of the plurality of regions.

CROSS-REFERENCE TO RELATED APPLICATION

This application claims benefit, under 35 U.S.C. §119(e), of U.S.Provisional Patent Application No. 61/052,934 filed on May 13, 2008which is incorporated herein, by reference, in its entirety.

FIELD OF THE APPLICATION

This application generally relates to systems and methods for medicalimaging.

BACKGROUND

Affecting more than one million Americans each year, skin cancer is themost prevalent form of cancer, accounting for nearly half of all newcancers reported, and the number is rising. However, according to theAmerican Academy of Dermatology, most forms of skin cancer are almostalways curable when found and treated early. For further details, see A.C. Geller et al., “The first 15 years of the American Academy ofDermatology skin cancer screening programs: 1985-1999,” Journal of theAmerican Academy of Dermatology 48(1), 34-41(2003), the entire contentsof which are hereby incorporated by reference herein. As the number ofpatients diagnosed with skin cancer continues to rise year-by-year,early detection and delineation are increasingly useful.

During a conventional examination, dermatologists visually survey theskin for lesions or moles that fit certain pre-defined criteria for apotential malignant condition. If an area is suspect, the doctor willperform a biopsy, sending the tissue to a pathology lab for diagnosis.Though effective, this method of detection is time consuming, invasive,and does not provide an immediate definitive diagnosis of a suspectlesion. It is also vulnerable to false positives that introduceunnecessary biopsy and associated costs. More importantly, earlydetection is very difficult at best, as developing cancers are notusually visible without close inspection of the skin.

Medical imaging has the potential to assist in the detection andcharacterization of skin cancers, as well as a wide variety of otherconditions.

Hyperspectral medical imaging is useful because, among other things, itallows a physician to obtain information about a patient that is notreadily visible to the naked eye. For example, a physician may be ableto visually identify the presence of a lesion, but may not be able tovisually determine the lesion's actual extent or what type of conditionit represents, or for that matter whether the lesion is benign orcancerous. Although the physician may be able to draw tentativeconclusions about the lesion based on some general visual indicatorssuch as color and shape, generally a biopsy is needed to conclusivelyidentify the type of lesion. Such a biopsy is invasive, painful, andpossibly unnecessary in cases where the lesion turns out to be benign.

In contrast, hyperspectral medical imaging is a powerful tool thatsignificantly extends the physician's ability to identify andcharacterize medical conditions. “Hyperspectral medical imaging” meansutilizing multiple spectral regions to image a subject, e.g., the entirebody or a body part of a human or animal, and thus to obtain medicalinformation about that subject. Specifically, each particular region ofa subject has a unique spectral signature extending across multiplebands of the electromagnetic spectrum. This spectral signature containsmedical, physiological, and compositional information about thecorresponding region of the subject. For example, if the subject has acancerous skin lesion, that lesion may have a different color, density,and/or composition than the subject's normal skin, thus resulting in thelesion having a different spectrum than the normal skin. While thesedifferences may be difficult to visually detect with the naked eye, thedifferences may become apparent through spectroscopic analysis, thusallowing the lesion (or other medical condition resulting in ameasurable spectroscopic feature) to be identified, characterized, andultimately more readily treated than would be possible usingconventional visual inspection and biopsy. Such spectral differences canbe presented to a user (such as a physician), for example, byconstructing a two-dimensional image of the lesion. See, for example,U.S. Pat. No. 6,937,885, the entire contents of which are incorporatedherein by reference. However, such an image can at times make itdifficult for the physician to identify exactly what part of thepatient's body generated that spectral information.

SUMMARY

Embodiments of the application provide systems and methods ofhyperspectral medical imaging.

Under one aspect, a method of displaying medical information about asubject that has a plurality of regions includes: resolving lightobtained from each region of the plurality of regions into acorresponding spectrum; selecting a portion of each spectrum, theselected portion including medical information about the correspondingregion; constructing an image based on the selected portion of eachspectrum; and projecting the image onto the subject.

Some embodiments further include generating the light with a lightsource. In some embodiments, the light has at least one of a broadbandspectrum and a narrowband spectrum. In some embodiments, the lightincludes at least one of an ultraviolet wavelength, a visiblewavelength, an infrared wavelength, and a terahertz wavelength. In someembodiments, resolving the light obtained from each region of theplurality of regions into a corresponding spectrum includes passing thelight into a spectrometer. In some embodiments, the spectrometerspatially separates the light into a plurality of component wavelengths,and records an intensity of each component wavelength of the pluralityof component wavelengths. In some embodiments, selecting the portion ofeach spectrum is based on at least one of: a spectral characteristic ofa predetermined medical condition, a spectral characteristic of apredetermined physiological feature, and a spectral characteristic of apredetermined chemical. In some embodiments, selecting the portion ofeach spectrum includes applying a digital filter to a digital signalrepresenting the spectrum. In some embodiments, selecting the portion ofeach spectrum includes applying at least one of a band-pass filter and aband-block filter to the spectrum. In some embodiments, constructing theimage includes assigning the selected portion of each spectrum to atleast one of a visible color and an intensity. In some embodiments,projecting the image onto the subject includes projecting the at leastone of the visible color and the intensity onto the subject. In someembodiments, the method further includes selecting a different portionof each spectrum, the different selected portion including differentmedical information about the corresponding region; and constructing anew image based on the different selected portion of each spectrum. Someembodiments include at least one of storing each spectrum and storingthe image. In some embodiments, each spectrum is stored in ahyperspectral data cube. In some embodiments, selecting a portion ofeach spectrum includes at least one of selecting a volume from thehyperspectral data cube and comparing information in the hyperspectraldata cube to known spectral information about a medical condition. Insome embodiments, the subject is a human. In some embodiments, there isa delay of less than about one minute between resolving light obtainedfrom each region of the plurality of regions and projecting the imageonto the subject.

Under another aspect, a system for displaying medical information abouta subject that has a plurality of regions includes: a spectrometer forresolving light obtained from each region of the plurality of regionsinto a corresponding spectrum; logic for selecting a portion of eachspectrum, the selected portion including medical information about thecorresponding region; logic for constructing an image based on theselected portion of each spectrum; and a projector for projecting theimage onto the subject.

Some embodiments further include a light source for irradiating thesubject with the light. In some embodiments, the light source generatesat least one of a broadband spectrum and a narrowband spectrum. In someembodiments, the light includes at least one of an ultravioletwavelength, a visible wavelength, an infrared wavelength, and aterahertz wavelength. In some embodiments, the spectrometer includes adiffraction grating for separating the light into a plurality ofcomponent wavelengths, and a sensor for recording an intensity of eachcomponent wavelength of the plurality of component wavelengths. In someembodiments, the logic selects the portion of each spectrum based on atleast one of: a spectral characteristic of a predetermined medicalcondition; a spectral characteristics of a predetermined physiologicalfeature; and a spectral characteristic of a predetermined chemical. Someembodiments further include a digital filter for digitally selecting theportion of a digital signal representing the spectrum. Some embodimentsfurther include at least one of a band-pass filter and a band-blockfilter for selecting the portion of each spectrum. In some embodiments,the logic for constructing the image assigns the selected portion ofeach spectrum to at least one of a visible color and an intensity. Insome embodiments, the projector projects the at least one of the visiblecolor and the intensity onto the subject. Some embodiments furtherinclude logic for: selecting a different portion of each spectrum, thedifferent selected portion including different medical information aboutthe corresponding region; and constructing a new image based on thedifferent selected portion of each spectrum. Some embodiments furtherinclude a storage medium for at least one of storing the image andstoring each spectrum. In some embodiments, the storage medium storeseach spectrum in a hyperspectral data cube. In some embodiments,selecting a portion of each spectrum includes at least one of selectinga volume from the hyperspectral data cube and comparing information inthe hyperspectral data cube to known spectral information about amedical condition. In some embodiments, the subject is a human. In someembodiments, there is a delay of less than about one minute between thespectrometer's resolution of light obtained from each region of theplurality of regions and the projector's projecting the image onto thesubject.

Under another aspect, a computer-readable medium storing a computerprogram executable by a computer for displaying medical informationabout a subject that has a plurality of regions includes instructionsfor: obtaining a spectrum corresponding to each region of the pluralityof regions; selecting a portion of each spectrum, the selected portionincluding medical information about the corresponding region;constructing an image based on the selected portion of each spectrum;and providing the image to a projection device for projection onto thesubject.

In some embodiments, the computer program further includes instructionsfor obtaining each spectrum from a spectrometer. In some embodiments,the computer program includes instructions for selecting the portion ofeach spectrum based on at least one of: a spectral characteristic of apredetermined medical condition, a spectral characteristic of apredetermined physiological feature, and a spectral characteristic of apredetermined chemical. In some embodiments, the computer programincludes instructions for selecting the portion of each spectrum byapplying a digital filter to a digital signal representing the spectrum.

In some embodiments, the computer program includes instructions forselecting the portion of each spectrum by applying at least one of aband-pass filter and a band-block filter to the spectrum. In someembodiments, the computer program includes instructions for constructingthe image by assigning the selected portion of each spectrum to at leastone of a visible color and an intensity. In some embodiments, thecomputer program further includes instructions for: selecting adifferent portion of each spectrum, the different selected portionincluding different medical information about the corresponding region;and constructing a new image based on the different selected portion ofeach spectrum. In some embodiments, the computer program furtherincludes instructions for at least one of storing each spectrum andstoring the image. In some embodiments, the computer program furtherincludes instructions for storing each spectrum in a hyperspectral datacube. In some embodiments, the computer program includes instructionsfor selecting a portion of each spectrum by at least one of selecting avolume from the hyperspectral data cube and comparing information in thehyperspectral data cube to known spectral information about a medicalcondition.

Under another aspect, a method of displaying medical information about asubject, the subject having a plurality of regions includes: resolvinglight obtained from each region of the plurality of regions into acorresponding spectrum; selecting a portion of each spectrum, theselected portion including medical information about the correspondingregion; constructing a spectral image based on the selected portion ofeach spectrum; combining the spectral image with other information aboutthe subject to form a composite image; and displaying the compositeimage.

In some embodiments, displaying the composite image comprises projectingthe composite image onto the subject. In some embodiments, displayingthe composite image comprises displaying the composite image on a videodisplay. In some embodiments, selecting the portion of each spectrum isbased on at least one of: a spectral characteristic of a predeterminedmedical condition, a spectral characteristic of a predeterminedphysiological feature, and a spectral characteristic of a predeterminedchemical. In some embodiments, the light includes at least one of anultraviolet wavelength, a visible wavelength, an infrared wavelength,and a terahertz wavelength. In some embodiments, the other informationabout the subject comprises an image of the subject. In someembodiments, the image of the subject is in at least one of anultraviolet band, a visible band, an infrared band, and a terahertzband.

Under another aspect, a system for displaying medical information abouta subject, the subject having a plurality of regions, includes: aspectrometer for resolving light obtained from each region of theplurality of regions into a corresponding spectrum; logic for selectinga portion of each spectrum, the selected portion including medicalinformation about the corresponding region; logic for constructing aspectral image based on the selected portion of each spectrum; logic forcombining the spectral image with other information about the subject toform a composite image; and a display for displaying the compositeimage.

In some embodiments, the display comprises a projector for projectingthe composite image onto the subject. In some embodiments, the displaycomprises a video display. In some embodiments, the logic selects theportion of each spectrum based on at least one of: a spectralcharacteristic of a predetermined medical condition, a spectralcharacteristic of a predetermined physiological feature, and a spectralcharacteristic of a predetermined chemical. Some embodiments furtherinclude a light source for irradiating the subject with the light. Insome embodiments, the light includes at least one of an ultravioletwavelength, a visible wavelength, an infrared wavelength, and aterahertz wavelength. Some embodiments further include an imager forobtaining an image of the subject, and wherein the other informationabout the subject comprises the image of the subject. In someembodiments, the image of the subject is in at least one of anultraviolet band, a visible band, an infrared band, and a terahertzband.

Under another aspect, a computer-readable medium stores a computerprogram executable by a computer for displaying medical informationabout a subject, the subject having a plurality of regions. The computerprogram includes instructions for: resolving light obtained from eachregion of the plurality of regions into a corresponding spectrum;selecting a portion of each spectrum, the selected portion includingmedical information about the corresponding region; constructing aspectral image based on the selected portion of each spectrum; combiningthe spectral image with other information about the subject to form acomposite image; and displaying the composite image.

In some embodiments, the computer program further comprises instructionsfor obtaining each spectrum from a spectrometer. In some embodiments,the computer program comprises instructions for displaying the compositeimage by projecting the composite image onto the subject. In someembodiments, the computer program comprises instructions for displayingthe composite image by displaying the composite image on a videodisplay. In some embodiments, the computer program comprisesinstructions for selecting the portion of each spectrum based on atleast one of: a spectral characteristic of a predetermined medicalcondition, a spectral characteristic of a predetermined physiologicalfeature, and a spectral characteristic of a predetermined chemical. Insome embodiments, the other information about the subject comprises animage of the subject at one of an ultraviolet wavelength, a visiblewavelength, an infrared wavelength, and a terahertz wavelength.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 illustrates a method for hyperspectral medical imaging, accordingto some embodiments.

FIG. 2 schematically illustrates a system for hyperspectral medicalimaging using real-time projection of spectral information onto asubject, according to some embodiments.

FIG. 3A schematically illustrates a hyperspectral data “plane” includingmedical information about a subject, according to some embodiments.

FIG. 3B schematically illustrates a hyperspectral data “cube” includingmedical information about a subject, according to some embodiments.

FIG. 4A schematically illustrates selection of a portion of ahyperspectral data “cube” including medical information about a subject,according to some embodiments.

FIG. 4B schematically illustrates a selected portion of a hyperspectraldata “cube” including medical information about a subject, according tosome embodiments.

FIG. 5 schematically illustrates an image based on a portion of aspectrum, according to some embodiments.

FIG. 6 schematically illustrates an embodiment of a processingsubsystem, according to some embodiments.

FIGS. 7A-7C are exemplary images from different spectral bands thatcontain different medical information about a subject.

DETAILED DESCRIPTION

Embodiments of the application provide systems and methods forhyperspectral medical imaging.

The present application provides systems and methods that enable aphysician to easily examine a subject by projecting spectral informationonto the subject and/or by displaying spectral information on a videodisplay. Specifically, the systems and methods include obtaininghyperspectral information from the subject, for example, by irradiatinga region of the subject with a light source, and collecting andspectrally analyzing the light from the subject. The systems and methodsinclude creating an image that maps the spectrally analyzed light ontovisible cues (such as false colors and/or intensity distributions) thatrepresent spectral features that include medical information about thesubject. In some embodiments, the systems and methods include projectingthose visible cues back onto the region of the subject in “real time”(that is, preferably with an imperceptible delay between irradiation andprojection). This allows the physician to concurrently orcontemporaneously inspect both the subject and the spectral informationabout the subject, which is represented by the visible cues that areprojected back upon the subject. The visible cues are projected directlyonto the regions of the subject having the spectral features upon whichthose visible cues are based and/or displayed on a video display.

Optionally, the projected and/or displayed image includes not only thevisible cues representing spectral information about the subject, butalso other types of information about the subject. For example, aconventional visible-light image of the subject (e.g., as recorded by aconventional video camera) can be obtained, and the spectral informationoverlaid on that conventional image in order to aid in correlationbetween the spectral features and the regions that generated thosefeatures. Or, for example, information can be obtained from multipletypes of sensors (e.g., LIDAR, color, thermal) and that informationcombined with the hyperspectral image, thus simultaneously providingdifferent, and potentially, complementary types of information about thesubject.

In some embodiments, the steps of obtaining light from the subject,processing that light to obtain an image, and projecting that image ontothe subject and/or displaying the image are performed with only a briefdelay between steps, so that the physician can view the images projectedonto the subject while he/she is examining the subject. For example, thedelay between obtaining the light and projecting the image may be lessthan about 1 ms, less than about 10 ms, less than about 100 ms, lessthan about 1 second, less than about 10 seconds, or less than about 1minute. Regardless of any delay between obtaining the light andprojecting and/or displaying the image, in some embodiments the obtainedlight and the projected/displayed light are separated from each other,either spectrally or temporally, in order to inhibit feedback of theprojected light into the system. To “spectrally” separate the obtainedand projected light, the projected light can be limited to a smallspectral range that is not used for spectral analysis; to “temporally”separate the obtained and projected light, the light is first obtainedfrom the subject, and the image then projected onto the subject withoutoverlapping between these two steps.

First, a brief overview of methods of hyperspectral medical imaging willbe provided. Then, systems for hyperspectral medical imaging will bedescribed in detail. The described methods and systems are merelyexemplary, and not limiting.

FIG. 1 provides an overview of a method 100 of hyperspectral medicalimaging, according to some embodiments.

First, a plurality of regions of the subject are irradiated with light(110). Collectively, the regions of the subject can include, forexample, a portion of one of the subject's body parts, an entire bodypart, multiple body parts, or the entire subject. However, eachindividual region may be quite small, e.g., less than 10 centimeters inarea, or less than 1 centimeter in area, or less than 100 millimeters inarea, or less than 10 millimeters in area, or less than 1 millimeter inarea, or less than 100 microns in area. Usefully, each individual regionis sufficiently small to allow resolution of the medical feature ofinterest, that is, so that a specified region containing the medicalfeature can be distinguished from other regions that do not contain thefeature. Different options for the source and spectral content of thelight are described in greater detail below.

Next, light is obtained from the regions of the subject (120). Dependingon the interactions between the regions of the subject and the spectrumof light with which they are irradiated, the light may be reflected,refracted, absorbed, and/or scattered from the regions of the subject.In some embodiments, one or more regions of the subject may even emitlight, e.g., fluoresce or photoluminesce in response to irradiation withthe light. A lens, mirror, or other suitable optical component can beused to obtain the light from the regions of the subject, as describedin greater detail below.

The light obtained from each region is then resolved into acorresponding spectrum (130). For example, the light obtained from eachregion can be passed into a spectrometer. The spectrometer includes adiffraction grating or other dispersive optical component that generatesa spatial separation between the light's component wavelengths. Thisspatial separation allows the relative intensities of the componentwavelengths in the spectrum to be obtained and recorded, e.g., using adetector such as a charge-coupled device (CCD) or other appropriatesensor that generates a digital signal representing the spectrum. Thedigital signal corresponding to each region can be stored, e.g., onreadable media or in random access memory. Examples of suitabledetectors include, but are no limited to, Si CCD, InGaAs, and HgCdTe.Suitable spectral ranges in some embodiments is 0.3 microns to 1 micron,0.4 micron to 1 micron, 1 micron to 1.7 microns, or 1.3 microns to 2.5microns. In some embodiments the detector contains between 320 and 1600spatial pixels. In other embodiments, the detector has more or lessspatial pixels. In some embodiments, the detector has a field of viewacross track that is between 14 degrees and 18.4 degrees. In someembodiments the detector samples at a rate of between 3 nm and 10 nm. Insome embodiments, the detector samples between 64 and 256 spectralbands. Of course, it is expected over time that improved detectors willbe devised and any such improved detector may be used in accordance withthe systems and methods of the present invention.

A portion of each spectrum is then selected (140). The selected portionincludes medical information about the corresponding region. Forexample, selecting the portion of each spectrum may include filteringthe spectrum based on the spectral characteristics of a predeterminedmedical condition, predetermined physiological feature, or predeterminedchemical (e.g., pharmaceutical compound). The selected portion of eachspectrum thus includes medical information about that region. Inembodiments in which digital signal representing the spectrum isgenerated, one or more portions of that digital signal can be modified(e.g., amplified or deleted) in order to select the desired portion.Exemplary algorithms for selecting portions of spectra are described ingreater detail below.

An image based on the selected portion of each spectrum is thenconstructed (150). The image includes information about the relativeintensities of selected wavelengths within the various regions of thesubject, and thus includes medical information about those regions. Theimage can represent the spectral information in a variety of ways. Forexample, the image may include a two-dimensional map that represents theintensity of one or more selected wavelengths within each region of thesubject. Such image can be monochromatic, with the intensity of the mapat a given region based on the intensity of the selected wavelengths(e.g., image intensity directly proportional to light intensity at theselected wavelengths). Alternately, the image can be colorful, with thecolor of the map at a given region based on the intensity of theselected wavelengths, or indices deducted from the selected wavelengths(more below). Although the image may represent information from one ormore non-visible regions of the electromagnetic spectrum (e.g.,infrared), the image itself is typically at least partially in thevisible range, so that it can be viewed by a physician or otherinterested party. Examples of images are provided below.

The image is then optionally combined or “fused” with other informationabout the subject (160). For example, the image can be overlaid on aconventional visible-light image of the subject, and/or can be combinedwith the output of other types of sensors. In some embodiments, spectraldata from one source, such as a hyperspectral image, is scaled to a greyscale or color whereas as spectral data from another completelyindependent source (e.g., x-rays, molecular resonance imaging, nuclearmagnetic resonance, a dynamic biomechanical skin measurement probe) thatis measured concurrently with the spectral image is topographicallyscaled to form a topographical or contour map. In such embodiments, thetopographical or contour map can be colored based on the grey scale orcolor scaled hyperspectral image data. Of course, the reverse is alsotrue, where the hyperspectral image data is converted to a topographicalor contour map and the spectral data from the independent spectralsource is normalized to a color scale or a grey scale which is then usedto color the topographical or contour map. Usefully, such a combined mapcan emphasize skin abnormalities that may not be apparent from any onesource of spectral data. For instance, if one spectral source flags aparticular region of the screen with a “red” result where red representsone end of the dynamic range of the sensor and another independentspectral source assigns a dense peak to this same region, where the peakrepresents the limits of the dynamic range of this independent spectralsource, the combined image from the two spectral sources will show apeak that is colored red. This can aid a physician in pinpointing aregion of interest. Two or more independent spectral sources can beused. In some embodiments, two or more, three or more, four or more,five or more spectral sources are combined into a single image. In someembodiments, some spectral sources are displayed in complementary(orthogonal) ways such as the example of a topographical map that iscolored based on the results of a different spectral source. In someembodiments, some spectral sources are combined using statisticaltechniques such as principal component analysis. In some embodiments,some spectral sources are combined in an additive manner. For example,in some embodiments, the corresponding pixel values of two spectralimages taken using different spectral sources are simply added togetherto form a fused image. Any such pixel by pixel based combination ofspectral images is within the scope of the present invention. In someembodiments, such spectral images taken using different spectroscopicmeans are taken concurrently so that the register of such images withrespect to the skin of the patient and to the respective spectral imagesis all known. In some embodiments, such spectral images are takensequentially but near in time with the assurance that the subject hasnot moved during the sequential measurements so that the images arereadily combined. In some embodiments, a skin registry technique is usedthat allows for the images from different spectral sources to be takenat different times and then merged together. For instance, a transparentgrid can be fastened to the subject's skin and the grid marks in each ofthe resultant spectra taken of the subject can be used to aligndifferent images together. Concurrently using different types of sensorsprovides a powerful way of obtaining rich information about the subject.Specific types of sensors and/or data fusion methods may be used toanalyze different types of targets. For example, in remote sensinganalysis, a sensor specific for submerged aquatic vegetation (SAV) hasbeen employed. Furthermore, normalized difference vegetation index(NDVI) is also developed for better representation. Similarly, inmedical imaging, specific sensors may be used to detect changes inspecific types of tissues, substances, or organs. Indices similar toNDVI can also be developed to normalize certain types of tissues,substances, or organs, either to enhance their presence or to reduceunnecessary background noise.

The rich information obtained by multi-sensor analyses may be integratedby data fusion methods in order to enhance image qualify or to addadditional information that is missing in the individual images. Imagefusion methods can be broadly classified into two categories: 1) visualdisplay transforms which involves the color composition of three bandsof imagery displayed in red-green-blue (RGB) or other colortransformations such as intensity-hue-saturation (IHS); and 2)statistical or numerical transforms based on channel statistics andinclude, for example, principal component analysis (PCA). Image fusionmethods that may be applied to the instant invention include, but arenot limited to, band overlay, high-pass filtering (HPF),intensity-hue-saturation (IHS) transformation, discrete wavelettransform (DWT), and principal component analysis (PCA).

Band Overlay. The band overlay (band substitution) is the simplest imagefusion technique. The major advantage of this technique is that there isno changes to the radiometric qualities of the data since there is noradiometric enhancement of the data. In some embodiments, the techniqueis used when the two sources are highly correlated. Panchromaticsharpening involves the substitution of the panchromatic band for themulti-spectral band covering the same region as the panchromatic band.The generation of color composite images is limited to the display ofonly three bands corresponding to the color guns of the display device(red-green-blue). As the panchromatic band has a spectral range coveringboth the green and red channels (PAN 0.50-0.75 mm; green 0.52-0.59 mm;red 0.62-0.68 mm), the panchromatic band can be used as a substitute foreither of those bands.

High-Pass Filtering Method (HPF). The HPF fusion method is a specificapplication of arithmetic techniques used to fuse imagery, whichinvolves use of arithmetic operations such as addition, subtraction,multiplication and division. HPF is an arithmetic technique that appliesa spatial enhancement filter to the high-resolution image before the twodata sets are merged together on a pixel-by-pixel basis. The HPF fusioncombines both spatial and spectral information using the band-additionapproach. It has been found that when compared to the IHS and PCA, theHPF method exhibits less distortion in the spectral characteristics ofthe data; and distortions are minimal and difficult to detect. Thisconclusion is based on statistical, visual and graphical analysis of thespectral characteristics of the data.

Intensity-Hue-Saturation (IHS). IHS transformation is one of the mostwidely used methods for merging complementary, multi-sensor data sets.The IHS transform provides an effective alternative to describing colorsby the red-green-blue display coordinate system. The possible range ofdigital numbers (DNs) for each color component is 0 to 255 for 8-bitdata. Each pixel is represented by a three-dimensional coordinateposition within the color cube. Pixels having equal components of red,green and blue lie on the grey line, a line from the cube to theopposite corner. The IHS transform is defined by three separate andorthogonal attributes, namely intensity, hue, and saturation. Intensityrepresents the total energy or brightness in an image and defines thevertical axis of the cylinder. Hue is the dominant or average wavelengthof the color inputs and defines the circumferential angle of thecylinder. It ranges from blue (0/360°) through green, yellow, red,purple, and then back to blue (360/0°). Saturation is the purity of acolor or the amount of white light in the image and defines the radiusof the cylinder. Of all methods to merge multi-spectral data, the IHSmethod distorts spectral characteristics the most and should be usedwith caution if detailed radiometric analysis is to be performed.Although IRS 1C LISS III acquires data in four bands, only three bandsare used for the study neglecting the fourth due to the poor spatialresolution. IHS transform is more successful in panchromatic sharpeningwith true color composites than when the color composites include nearor mid-infrared bands.

Principal Component Analysis (PCA). The PCA is a commonly used tool forimage enhancement and the data compression. The originalinter-correlated data are mathematically transformed into new,uncorrelated images called components or axes. The procedure involves alinear transformation so that the original brightness values arere-projected onto a new set of orthogonal axes. PCA is a relevant methodfor merging remotely sensed imagery because of its ability to reduce thedimensionality of the original data from n to 2 or 3 transformedprincipal component images, which contains the majority of theinformation from the original spectroscopic data sources. For example,PCA can be used to merge several bands of multispectral data with onehigh spatial resolution band. Image fusion can be done in two ways usingthe PCA. The first method is very similar to IHS transformation. Thesecond method involves a forward transformation that is performed on allimage channels from the different sensors combined to form one singleimage file.

Discrete Wavelet Transform (DWT). The DWT method involves waveletdecomposition where wavelet transformation converts the images intodifferent resolutions. Wavelet representation has both the spatial andfrequency components. Exemplary approaches for wavelet decompositionincludes the Mallat algorithm which can use wavelet function such as theDaubechies functions (db1, db2, . . . ) and the á Trous algorithm whichmerges dyadic wavelet and non_dyadic data in a simple and efficientprocedure. There are two approaches for image fusion based on waveletdecomposition: the substitution method and the additive method. In thesubstitution method, after the wavelet coefficients of multispectral andpanchromatic images are obtained, some wavelet coefficients ofmultispectral image are substituted by wavelet coefficients ofpanchromatic image followed by an inverse wavelet transform. In theadditive method, wavelet planes of the panchromatic image may beproduced and added to the red, green, and blue bands directly or to theintensity component which is extracted from the red, green, and bluebands. In some embodiments, a transformation step may be used to convertthe HIS component (with a new intensity) into new R,G,B data.

More detailed description of the exemplary imagery fusion methods can befound in, for example, Harris et al., 1990, “IHS transform for theintegration of radar imagery with other remotely sensed data,”Photogrammetric Engineering and Remote Sensing, 56(12): 1631-1641; Pholand van Genderen, 1998, “Multisensor image fusion in remote sensing:concepts, methods and applications,” International Journal of RemoteSensing, 19(5): 823-854; Chavez et al., 1991, “Comparison of threedifferent methods to merge multi-resolution and multi-sectoral data:Landsat TM and SPOT Panchromatic,” Photogrammetric Engineering andRemote Sensing, 57(3): 295-303; Pellemans et al., 1993, “Mergingmultispectral and panchromatic SPOT images with respect to radiometricproperties of the sensor,” Photogrammetric Engineering and RemoteSensing, 59(1): 81-87; Nunez et al., 1999, “Multiresolution based imagefusion with additive wavelet decomposition,” IEEE Transactions onGeoscience and Remote Sensing, 37(3): 1204-1211; Steinnocher, 1997,“Applications of adaptive filters for multisensoral image fusion,”Proceedings of the International Geoscience and Remote Sensing Symposium(IGARASS '97), Singapore, August 1997, 910-912; and Chavez and Kwarteng,1989, “Extracting spectral contrast in Landsat Thematic Mapper imagedata using selective principal component analysis,” PhotogrammetricEngineering and Remote Sensing, 55(3): 339-348, each of which is herebyincorporated by reference herein in its entirety.

For example, as illustrated in FIGS. 7A-7C, different regions of theelectromagnetic spectrum contain significantly different informationabout a subject. FIG. 7A is an image of a subject obtained in thevisible portion of the spectrum (e.g., is a conventional video orphotographic image of the subject). FIG. 7B is an image of the samesubject, but obtained in the thermal portion of the spectrum (e.g., SWIRto MIR). FIG. 7C is another image of the same subject but obtained instill another portion of the spectrum. The different images wereobtained with appropriate conventional sensors that are known in theart, and highlight different aspects of the medical condition of thesubject. By obtaining relevant information in the appropriateelectromagnetic band(s), and combining that information with an imagerepresenting spectral information about the subject such as thatdescribed herein, images can be generated that provide significantlymore detailed information than an image that represents only a singletype of information.

Referring again to FIG. 1, the image, which optionally also containsother information about the subject, is either projected onto thesubject (170) or displayed on a video display (180). In embodiments inwhich the image is projected onto the subject (170), the regions of theimage corresponding to regions of the subject are projected directly, orapproximately directly, onto those regions of the subject. This allows aphysician to concurrently or contemporaneously inspect the physicalregions of the subject as well as the image, which is a visiblerepresentation of selected spectral features generated by those physicalregions. This allows the physician to easily correlate those spectralfeatures with physical features of the subject, thus aiding in thediagnosis and treatment of a medical condition.

Alternately, in embodiments in which the image is displayed on a videodisplay (180), the physician can inspect the image, optionally while heis physically examining the subject, and thus obtain information that isuseful in diagnosing and treating a medical condition. A normal (visiblelight) image of the regions of the subject is displayed underlying theimage containing spectral information, which can aid the physician'sability to correlate the spectral features with physical features of thesubject. In some embodiments, the image is both projected onto thesubject and displayed on a video monitor.

In some embodiments, the image and/or the spectra are stored for laterprocessing. For example, storing an image of a lesion each time thesubject is examined can help the physician track the growth of thelesion and/or its response to treatment. Storing the spectra can enableother information to be obtained from the spectra at a later time, forexample, if a certain new kind of tumor is identified and the spectraanalyzed to see if the subject has the new type of tumor. An image canbe generated based on the second analysis.

FIG. 2 schematically illustrates a hyperspectral medical imaging system200 using real-time projection of spectral information onto a subject,according to some embodiments. In FIG. 2, the subject is represented asan area 201 that includes a plurality of regions 201′, which areillustrated as a plurality of small squares. The area 201 can be one ofthe subject's body parts or a portion thereof (e.g., a selected area ofthe subject's skin), can be multiple body parts or portions thereof, orcan even be the entire subject. The plurality of regions 201′ aresubsets of area 201. The regions 201′ need not be directly adjacent oneanother, and need not be square, or even regularly shaped. The regions201′ collectively represent a sampling of the area 201 that is to becharacterized. In the illustrated embodiment, the regions 201′ areorganized into rows 202 and columns 203 of regions. The subject is, ofcourse, not considered to be part of the imaging system.

The hyperspectral imaging system 200 includes an illumination subsystem210, a sensor subsystem 230, a processor subsystem 250, and a projectionsubsystem 270. The processor subsystem 250 is in operable communicationwith each of the illumination, sensor, and projection subsystems 270,and coordinates the operations of these subsystems in order to irradiatethe subject, obtain spectral information from the subject, construct animage based on the spectral information, and project the constructedimage onto the subject. Specifically the illumination subsystem 210irradiates with light each region 201′ within area 201 of the subject,which light is represented by the dotted lines. The light interacts withthe plurality of regions 201′ of the subject. The sensor subsystem 230collects light from each region of the plurality of regions 201′ of thesubject, which light is represented by the dashed lines. The sensorsubsystem 230 resolves the light from each region 201′ into acorresponding spectrum, and generates a digital signal representing thespectra from all the regions 201′. The processor subsystem 250 obtainsthe digital signal from the sensor subsystem 230, and processes thedigital signal to generate an image based on selected portions of thespectra that the digital signal represents. The processor subsystem 250then passes that image to projection subsystem 270, which projects theimage onto the plurality of regions 201′ of the subject, the light fromwhich is represented by the dash-dot lines.

Each of the subsystems 210, 230, 250, and 270 will now be described ingreater detail.

Illumination Subsystem

Illumination subsystem 210 generates light having a spectrum thatincludes a plurality of component wavelengths. The spectrum can includecomponent wavelengths in the ultraviolet (UV) band (in the range ofabout 10 nm to about 400 nm); visible band (in the range of about 400 nmto about 700 nm); near infrared (NIR) band (in the range of about 700 nmto about 2500 nm); mid-wave infrared (MWIR) band (in the range of about2500 nm to about 10 μm); long-wave infrared (LWIR) band (in the range ofabout 10 μm to about 100 μm); and/or terahertz (THz) band (in the rangeof about 100 μm to about 1 mm), among others. The NIR, MWIR, and LWIRare collectively referred to herein as the infrared (IR) band. The lightcan include a plurality of component wavelengths within one of thebands, e.g., a plurality of wavelengths in the NIR band, or in the THz.Alternately, the light can include one or more component wavelengths inone band, and one or more component wavelengths in a different band,e.g., some wavelengths in the visible, and some wavelengths in the IR.Light with wavelengths in both the visible and NIR bands is referred toherein as “VNIR.” Other useful ranges may include the region 1,000-2,500nm (shortwave infrared, or SWIR).

The illumination subsystem 210 generates the light using one or morelight sources. For example, the illumination subsystem 210 can include asingle broadband light source, a single narrowband light source, aplurality of narrowband light sources, or a combination of one or morebroadband light source and one or more narrowband light source. By“broadband” it is meant light that includes component wavelengths over asubstantial portion of at least one band, e.g., over at least 20%, or atleast 30%, or at least 40%, or at least 50%, or at least 60%, or atleast 70%, or at least 80%, or at least 90%, or at least 95% of theband, or even the entire band, and optionally includes componentwavelengths within one or more other bands. A “white light source” isconsidered to be broadband, because it extends over a substantialportion of at least the visible band. By “narrowband” it is meant lightthat includes components over only a narrow spectral region, e.g., lessthan 20%, or less than 15%, or less than 10%, or less than 5%, or lessthan 2%, or less than 1%, or less than 0.5% of a single band. Narrowbandlight sources need not be confined to a single band, but can includewavelengths in multiple bands. A plurality of narrowband light sourcesmay each individually generate light within only a small portion of asingle band, but together may generate light that covers a substantialportion of one or more bands, e.g., may together constitute a broadbandlight source.

One example of a suitable light source for use in illumination subsystem210 is a diffused lighting source that uses a halogen lamp, such as theLowel Pro-Light Focus Flood Light. A halogen lamp produces an intensebroad-band white light which is a close replication of daylightspectrum. Other light sources that can be used in illumination subsystem210 include a xenon lamp, a hydrargyrum medium-arc iodide lamp, and/or alight-emitting diode. Other types of light sources are also suitable.

Depending on the particular light source(s) used, illumination subsystem210 can generate light in which the relative intensities of itscomponent wavelengths are uniform (e.g., are substantially the sameacross the spectrum), or vary smoothly as a function of wavelength, orare irregular (e.g., in which some wavelengths have significantly higherintensities than slightly longer or shorter wavelengths), and/or canhave gaps. Alternatively, the spectrum can include one or morenarrow-band spectra in regions of the electromagnetic spectrum that donot overlap with each other.

In some embodiments, illumination subsystem 210 substantially uniformlyirradiates regions 201′ with light. That is, the intensity of light atone region 201′ is substantially the same as the intensity of light atanother region 201′. In other embodiments, the intensity of the lightvaries from one region 201′ to the next.

Illumination subsystem 210 is useful because it irradiates regions 201′with light of sufficient intensity to enable sensor subsystem 230 toobtain sufficiently high quality spectra from those regions 201′, thatis, that a spectrum with a sufficient signal-to-noise ratio can beobtained from each region 201′ to be able to obtain medical informationabout each region 201′. However, in some embodiments, ambient light,such as fluorescent, halogen, or incandescent light in the room, or evensunlight, is a satisfactory source of light. In such embodiments, theillumination subsystem 210 is not activated, or the system may not eveninclude illumination system 210. Sources of ambient light typically donot communicate with the processing subsystem 250, but instead operateindependently of system 200.

The light from illumination subsystem 210 (illustrated as dotted linesin FIG. 2) interacts with the plurality of regions 201′ within area 201.The interaction between the light and each region 201′ depends on theparticular physiological structure and characteristics of that region.The particular interactions between the light and each individualirradiated region of the subject impart a spectral signature onto thelight obtained from that region. This spectral signature can be used toobtain medical information about the subject. Specifically, differentregions interact differently with the light depending on the presenceof, for example, a medical condition in the region, the physiologicalstructure of the region, and/or the presence of a chemical in theregion. For example, fat, skin, blood, and flesh all interact withvarious wavelengths of light differently from one another. Similarly, agiven type of cancerous lesion interacts with various wavelengths oflight differently from normal skin, from non-cancerous lesions, and fromother types of cancerous lesions. A given chemical that is present(e.g., in the blood, or on the skin) interacts with various wavelengthsof light differently from other types of chemicals. Thus, the lightobtained from each irradiated region of the subject has a spectralsignature based on the characteristics of the region, which signaturecontains medical information about that region.

For example, the structure of skin, while complex, can be approximatedas two separate and structurally different layers, namely the epidermisand dermis. These two layers have very different scattering andabsorption properties due to differences of composition. The epidermisis the outer layer of skin. It has specialized cells called melanocytesthat produce melanin pigments. Light is primarily absorbed in theepidermis, while scattering in the epidermis is considered negligible.For further details, see G. H. Findlay, “Blue Skin,” British Journal ofDermatology 83(1), 127-134 (1970), the entire contents of which arehereby incorporated by reference herein.

The dermis has a dense collection of collagen fibers and blood vessels,and its optical properties are very different from that of theepidermis. Absorption of light of a bloodless dermis is negligible.However, blood-borne pigments like oxy- and deoxy-hemoglobin and waterare major absorbers of light in the dermis. Scattering by the collagenfibers and absorption due to chromophores in the dermis determine thedepth of penetration of light through skin.

In the visible and near-infrared (VNIR) spectral range and at lowintensity irradiance, and when thermal effects are negligible, majorlight-tissue interactions include reflection, refraction, scattering andabsorption. For normal collimated incident radiation, the regularreflection of the skin at the air-tissue interface is typically onlyaround 4%-7% in the 250-3000 nanometer (nm) wavelength range. Forfurther details, see R. R. Anderson and J. A. Parrish, “The optics ofhuman skin,” Journal of Investigative Dermatology 77(1), 13-19 (1981),the entire contents of which are hereby incorporated by referenceherein. When neglecting the air-tissue interface reflection and assumingtotal diffusion of incident light after the stratum corneum layer, thesteady state VNIR skin reflectance can be modeled as the light thatfirst survives the absorption of the epidermis, then reflects backtoward the epidermis layer due the isotropic scattering in the dermislayer, and then finally emerges out of the skin after going through theepidermis layer again.

Using a two-layer optical model of skin, the overall reflectance can bemodeled as:

R(λ)=T _(E) ²(λ)R _(D)(λ)

where T_(E)(λ) is the transmittance of epidermis and R_(D)(λ) is thereflectance of dermis. The transmittance due to the epidermis is squaredbecause the light passes through it twice before emerging out of skin.Assuming the absorption of the epidermis is mainly due to the melaninconcentration, the transmittance of the epidermis can be modeled as:

T _(E)(λ)=exp(d _(E) c _(m) m(λ)),

where d_(E) is the depth of the epidermis, c_(m) is the melaninconcentration and μ_(a)(λ) is the absorption coefficient function formelanin. For further details, see S. L. Jacques, “Skin optics,” OregonMedical Laser Center News Etc. (1988), the entire contents of which arehereby incorporated by reference herein.

The dermis layer can be modeled as a semi-infinite homogeneous medium.The diffuse reflectance from the surface of dermis layer can be modeledas:

${{R_{D}(\lambda)} = {\exp\left( \frac{- A}{\sqrt{3\left( {1 + \frac{\mu_{s}(\lambda)}{\mu_{a}(\lambda)}} \right)}} \right)}},$

where constant A is approximately 7-8 for most soft tissues, and NO isthe overall absorption coefficient function of the dermis layer. Forfurther details, see S. L. Jacques, “Diffuse reflectance from asemi-infinite medium,” Oregon Medical Laser News Etc. (1999), the entirecontents of which are hereby incorporated by reference herein. The termμ_(a)(λ) can be approximated as:

λ_(a)(λ)=c _(o) o(λ)+c _(h) h(λ)+c _(w) w(λ),

where c_(o), c_(h), and c_(w) are the concentrations of oxy-hemoglobin,deoxy-hemoglobin and water, respectively, while o(λ), h(λ), and w(λ) arethe absorption coefficient functions of oxy-hemoglobin,deoxy-hemoglobin, and water, respectively. For further details, see S.Wray et al., “Characterization of the near infrared absorption spectraof cytochrome aa3 and haemoglobin for the non-invasive monitoring ofcerebral oxygenation,” Biochimica et Biophysica Acta 933(1), 184-192(1988), the entire contents of which are hereby incorporated byreference herein. The scattering coefficient function for soft tissuecan be modeled as:

μ_(s)(λ)=αλ^(−b),

where a and b depend on the individual subject and are based, in part,on the size and density of collagen fibers and blood vessels in thesubject's dermis layer.

From the above equations, for a fixed depth of epidermis layer, the skinreflectance R(λ) can be modeled as a function ƒ of seven parameters:

R(λ)=ƒ(a,b,c _(m) ,c _(o) ,c _(h) ,c _(w),λ)

where a, b, c_(m), c_(o), c_(h), and c_(w), are as described above. Theskin reflectance R(λ) may also depend on other variables not listedhere. For example, long wavelengths (e.g., in the MWIR, FIR, or THzbands) may interact weakly with the surface of the skin and interactstrongly with fat, flesh, and/or bone underlying the skin, and thereforevariables other than those discussed above may be relevant.

The value of the skin's reflectance as a function of wavelength, R(λ),can be used to obtain medical information about the skin and itsunderlying structures. For example, when skin cancers like basal cellcarcinoma (BCC), squamous cell carcinoma (SCC), and malignant melanoma(MM) grow in the skin, the molecular structure of the affected skinchanges. Malignant melanoma is a cancer that begins in the melanocytespresent in the epidermis layer. For further details, see “Melanoma SkinCancer,” American Cancer Society (2005), the entire contents of whichare hereby incorporated by reference herein. Most melanoma cells producemelanin that in turn changes the reflectance characteristics as afunction of wavelength R(λ) of the affected skin. Squamous and basalcells are also present in the epidermis layer. The outermost layer ofthe epidermis is called the stratum corneum. Below it are layers ofsquamous cells. The lowest part of the epidermis, the basal layer, isformed by basal cells. Both squamous and basal cell carcinomas producecertain viral proteins that interact with the growth-regulating proteinsof normal skin cells. The abnormal cell growth then changes theepidermis optical scattering characteristics and consequently the skinreflectance properties as a function of wavelength R(λ). Thus,information about different skin conditions (e.g., normal skin, benignskin lesions and skin cancers) can be obtained by characterizing thereflectance R(λ) from the skin. This can be done, for example, using thesensor subsystem 230 and processor subsystem 250, as described ingreater detail below.

Sensor Subsystem

The sensor subsystem 230 obtains light from each region 201′ andresolves that light into a corresponding spectrum. In some embodiments,the sensor subsystem 230 includes a lens 232 that collects light from aregion 201′, an optional slit 233 that selects a portion of thecollected light, a dispersive optic 234 that spatially separates thelight into a plurality of component wavelengths, a charge-coupled device(CCD) 236 that records an intensity of each component wavelength of theplurality of component wavelengths (e.g., the spectrum of the region201′), a sensor control subsystem 238, and storage device 240 forstoring spectra. Storage device can be volatile (e.g., RAM) ornon-volatile (e.g., a hard disk drive).

The lens 232 captures at least a portion of the light from each regionof the plurality of regions 201′, as represented by the dashed lines.The optional slit 233 selects a portion of the light captured by thelens 232. For example, in an embodiment described more fully below, aslit can be used in combination with a scanning optic to sequentiallyselect lines 202 of regions of the subject.

The light obtained from each region of the plurality of regions 201′ isthen directed onto dispersive optic 234. The dispersive optic 234 canbe, for example, a diffractive optic such as transmission grating (e.g.,a phase grating or an amplitude grating) or a reflective grating, or aprism or similar dispersive optic. The dispersive optic 234 spatiallyseparates the different component wavelengths of the obtained light,allowing the intensity of each of the component wavelengths (thespectrum) to be obtained for each region 201′.

The CCD 236 is arranged at a fixed distance from the dispersive optic234. The distance between the CCD 236 and the dispersive optic 234,together with the size of the sensor elements that make up the CCD 236,determines (in part) the spectral resolution of the sensor subsystem230. The spectral resolution, which is the width (e.g., full width athalf maximum, or FWHM) of the component wavelengths collected by thesensor element, is selected so as to be sufficiently small to capturespectral features of medical conditions of interest. The sensedintensity of component wavelengths depends on many factors, includingthe light source intensity, the sensor element sensitivity at eachparticular component wavelength, and the exposure time of the sensorelement to the component wavelength. These factors are selected suchthat the sensor subsystem 230 is capable of sufficiently determining theintensity of component wavelengths that it can distinguish the spectralfeatures of medical conditions of interest.

Under control of the sensor control subsystem 238, the CCD 236 sensesand records the intensity of each of the component wavelengths (thespectrum) from each region 201′ in the form of a digital signal. In someembodiments, the sensor control subsystem stores the digital signal intostorage device 240. The sensor control subsystem 238 may be integratedwith the CCD 236, or may be in operable communication with the CCD 236.Collectively, the dispersive optic 234 and CCD 236 form a spectrometer(which can also include other components). Note that the efficiency of adispersive optic and the sensitivity of a CCD can bewavelength-dependent. Thus, the dispersive optic and CCD can be selectedso as to have satisfactory performance at all of the wavelengths ofinterest to the measurement (e.g., so that together the dispersive opticand CCD allow a sufficient amount of light to be recorded from which aspectrum can be obtained).

The light need not be obtained and/or spectrally resolved concurrentlyfrom all regions 201′. For example, the light from each individualregion 201′ can be obtained separately. Or, for example, the light froma subset of the regions can be obtained concurrently, but at a differenttime from light from other subsets of the regions. Or, for example, aportion of the light from all the regions can be obtained concurrently,but at a different time from other portions of the light from all theregions (for example, the intensity of a particular wavelength from allregions can be measured concurrently, and then the intensity of adifferent wavelength from all regions can be measured concurrently). Insome embodiments, light is obtained from a single row 202 at a time, ora single column 203 at a time.

One example of a suitable sensor subsystem 230 is the AISA hyperspectralsensor, which is an advanced imaging spectrometer manufactured by Specim(Finland). The AISA sensor measures electromagnetic energy over thevisible and NIR spectral bands, specifically from 430 nm to 910 nm. TheAISA sensor includes a “push broom” type of sensor, meaning that itscans a single line at a time, and has a spectral resolution of 2.9 nmand a 20 degree field of vision.

During operation, the AISA hyperspectral sensor obtains light from asingle row 202 of the regions 201′ at a time. The AISA sensor spectrallyresolves the light from each of the regions 201′ in that row 202 using adispersive optic. FIG. 3A schematically illustrates the resolution ofthe spectrum of each region 201′ in a row 202 into a “hyperspectral dataplane” 305. The plane 305 includes a plurality of columns 301′, each ofwhich includes the spectrum of a corresponding region 201′. As FIG. 3Aillustrates, the intensity of the spectrum within each column 301′varies as a function of wavelength. This intensity variation is a resultof the light's wavelength-dependent interaction with the correspondingregion 201′ of the subject, and thus contains medical information aboutthat region 201′. For example, using the model described above, thespectrum can be modeled as a wavelength-dependent reflectance R(λ) thatis a function of several variables, e.g., the concentrations of melanin,oxy-hemoglobin, deoxy-hemoglobin and water. In the illustratedembodiment, a dark color at a given wavelength means less reflection oflight from the region 201′ (e.g., strong absorption of that wavelengthby the region 201′, such as due to a high concentration of melanin) anda light color at a given wavelength means more reflection of light fromthe region 201′ (e.g., weak absorption of that wavelength by the region201′, such as due to a low concentration of melanin) Thus, in FIG. 3,the plane 305 indicates that the left-most columns 301′ had a relativelyhigh reflection at long wavelengths, which reflects the fact that theleft-most regions 201′ of row 202 contain different medical informationthan the right-most regions 201 of row 202.

After obtaining the plane 305 for row 202, the AISA sensor provides theplane 305 to the processor subsystem 250. The AISA sensor thensequentially obtains additional planes 305 for the other rows 202 withinarea 201. In some embodiments, the AISA sensor sequentially obtainsadditional planes 305 by rotating a scanning mirror (not shown) that ismounted in front of lens 232. The scanning mirror directs light fromsequential regions of the subject into the sensor for analysis. In otherembodiments, the AISA sensor sequentially obtains light from additionalregions by moving relative the subject, or by the subject movingrelative to the sensor. Other mechanisms can be used to scan sequentialregions of the subject, such as the focal plane scanner described inYang et al., “A CCD Camera-based Hyperspectral Imaging System ofStationary and Airborne Applications,” Geocarto International, Vol. 18,No. 2, June 2003, the entire contents of which are incorporated byreference herein.

FIG. 3B illustrates a “hyperspectral data cube” 306 that the AISA sensorconstructs using the planes 305 obtained for each of the rows 202 withinarea 201. The cube 306 includes a spectrum 307 corresponding to eachregion 201′. The spectra are stored within a three-dimensional volume,in which two of the axes represent the x- and y-coordinates of theregions 201′, and the third axis represents the wavelengths within thecorresponding spectra. The intensity at a particular point within thecube 306 represents the intensity of a particular wavelength (λ) at aparticular region 201′ having coordinates (x, y). The AISA sensor storescube 306 in storage device 240. The spectra corresponding to the regions201′ can, of course, be stored in any other suitable format. Sensorsother than an AISA sensor can also obtain hyperspectral data planes andcubes.

Other types of sensors are also suitable, such as a liquid crystaltunable filter (LCTF) based hyperspectral sensor. An LCTF-based sensorobtains light from all regions 201′ at a time, within a single narrowband at a time. The LCTF-based sensor selects the single band byapplying an appropriate voltage to the liquid crystal tunable filter,and recording a map of the reflected intensity of the regions 201′ atthat band. The LCTF-based sensor then sequentially selects differentbands by appropriately adjusting the applied voltage, and recordingcorresponding maps of the reflected intensity of the regions 201′ atthose bands. Another suitable type of sensor is a “whisk-broom” sensorthat simultaneously collects spectra from both columns and rows ofregions 201′ in a pre-defined pattern.

Processor Subsystem

Referring again to FIG. 2, the processor subsystem 250 includes astorage device 252, a spectral portion selector 254, and an imageconstructor 256. The processor subsystem 250 obtains from sensorsubsystem 230 a spectrum corresponding to each region 201′ of area 201,and stores the spectrum in storage device 252, which can be volatile(e.g., RAM) or non-volatile (e.g., a hard disk drive). Optionally, thespectra are arranged in a hyperspectral data plane or cube, such asthose described in greater detail above. Based on information stored instorage device 252, logic operating in the spectral portion selector 250selects a portion of each spectrum; the selected portion includesmedical information about the corresponding region. Then, logicoperating in the image constructor 256 constructs an image based on theselected portion of each spectrum. Optionally, the image constructor 256combines the image with other information about the subject, e.g.,images obtained in other electromagnetic bands.

In some embodiments, the spectral portion selector 254 selects thespectral portion based on one or more spectral characteristics of apre-determined medical condition. For example, as noted above, thereflectance R(λ) in the VNIR band of a given region of skin can bemodeled as a function ƒ of several parameters. Certain portions of thatreflectance R(λ) may contain indicia of that medical condition. Byselecting spectral portions that potentially include these indicia, thepresence or absence of that condition can be determined for thecorresponding region.

The spectral portion selector 254 is not limited to selecting spectralregions based on the spectral characteristics of only a singlepre-determined condition at a time, but instead can select multiplespectral regions based on multiple pre-determined conditions. Forexample, as noted above, a physician may not be able to determinethrough visual inspection whether a lesion is benign or cancerous. Thusit can be useful for the spectral portion selector 254 to selectspectral regions based on spectral characteristics of a wide variety ofpotential conditions.

In one example, a particular medical condition has identifiable spectralcharacteristics within a narrow, contiguous wavelength range 21-22(e.g., 850-900 nm). The spectral portion selector 254 selects the range21-22 by applying a filter, e.g., a band-pass filter and/or a band-blockfilter, to each spectrum. The filter can be digital or analog, and canbe smooth (e.g., Gaussian) or can have sharp edges. In embodiments inwhich a hyperspectral data cube is generated, the spectral portionselector 254 selects portions of the cube that fall within the desiredwavelength range. Multiple spectral regions can also be selected, andneed not be contiguous with one another. The unused spectral portionsneed not be discarded, but can be saved in storage 252 for later use.For example, the system may later obtain information from a differentportion of the spectrum.

FIG. 4A illustrates an embodiment in which the spectra of the differentregions 201′ are stored in a hyperspectral data cube 405, and thespectral portion selector 254 selects the wavelength region λ₁-λ₂associated with the condition by selecting a volume 406 from the cube405. The boundaries of volume 406 are defined by the x- and y-dimensionsof area 201 and by wavelength range λ₁-λ₂. FIG. 4B illustrates aselected volume 406. The intensity distribution at the top face 410 ofthe volume corresponds to the spectral intensity at wavelength λ₁ ofeach region 201′ within the area 201, while the intensity distributionat the bottom face (not shown) of the volume corresponds to the spectralintensity at wavelength 22. Thus it can be seen that regions in thelower left corner of the area 201 strongly interacted with light atwavelength λ₁, while regions in the upper right corner of the area 201weakly interacted with light at wavelength λ₁. This indicates that themedical condition is present in the regions in the lower left corner ofarea 201, but not in the regions in the upper right corner of area 201.

After the spectral portion selector 254 selects a portion of eachspectrum, the image constructor 256 constructs an image based on theselected portion of each spectrum. Specifically, the image constructor256 creates a representation (e.g., a 2D or 3D representation) ofinformation within the selected portions of the spectra. In one example,the image constructor 256 constructs a two-dimensional intensity map inwhich the spatially-varying intensity of one or more particularwavelengths (or wavelength ranges) within the selected spectral portionsis represented by a corresponding spatially varying intensity of avisible marker. FIG. 5 illustrates an image 510 that is based on thespatial variations in intensity at wavelength λ₁ that are illustrated inFIG. 4B. The image 510 includes regions 511, 512, and 513 of increasingintensity, respectively, which represent the magnitude of interaction ofdifferent regions 201′ with light at wavelength λ₁. While FIG. 5 ismonochromatic, false colors can also be assigned to represent differentintensities or other information. For example, in embodiments in whichmultiple spectral portions corresponding to multiple potentialconditions are selected, spectral portions corresponding to onecondition can be assigned one color, and spectral portions correspondingto another condition can be assigned a different color, thus allowingthe physician or other interested part to readily distinguish areasaffected by the different conditions.

FIG. 6 schematically illustrates an exemplary embodiment of processorsubsystem 250. The subsystem 250 includes a computer system 10 having:

-   -   a central processing unit 22;    -   a main non-volatile storage unit 14, for example a hard disk        drive, for storing software and data, the storage unit 14        controlled by storage controller 12;    -   a system memory 36, preferably high speed random-access memory        (RAM), for storing system control programs, data, and        application programs, including programs and data loaded from        non-volatile storage unit 14; system memory 36 may also include        read-only memory (ROM);    -   a user interface 32, including one or more input devices (e.g.,        keyboard 28, a mouse) and a display 26 or other output device;    -   a network interface card 20 (communications circuitry) for        connecting to any wired or wireless communication network 34        (e.g., a wide area network such as the Internet);    -   a power source 24 to power the aforementioned elements; and    -   an internal bus 30 for interconnecting the aforementioned        elements of the system.

Operation of computer 10 is controlled primarily by operating system640, which is executed by central processing unit 22. Operating system640 can be stored in system memory 36. In some embodiments, systemmemory 36 also includes:

-   -   a file system 642 for controlling access to the various files        and data structures used herein;    -   the spectral portion selector 254 described above;    -   the image constructor 256 described above;    -   the measured hyperspectral cube 644, which includes a plurality        of measured hyperspectral data planes;    -   medical condition spectral characteristics 646 (more below);    -   the selected portion of the measured hyperspectral data cube        660; and    -   the constructed image based on the selected portion of the        measured hyperspectral data cube 670.

The measured hyperspectral data cube 644, the portion selected thereof660, and the constructed image based thereon 670 need not all beconcurrently present, depending on which stages of the analysis thatprocessor subsystem 250 has performed.

As illustrated in FIG. 6, computer 10 includes medical conditionspectral characteristics 646, which includes spectral information 648for a plurality of medical conditions, “Condition 1” through “ConditionM.” The spectral information for each condition includes a set ofspectral characteristics 654 that the spectral portion selector 254 canuse to determine whether the region corresponding to the measuredhyperspectral data cube 644 has condition 1. In some embodiments, thespectral characteristics 646 are stored in a single database. In otherembodiments, such data is instead stored in a plurality of databasesthat may or may not all be hosted by the same computer 10. In suchembodiments, some of the data illustrated in FIG. 6 as being stored inmemory 36 is stored on computer systems that are not illustrated by FIG.6 but that are addressable by wide area network 34.

In some embodiments, the data illustrated in memory 36 of computer 10 ison a single computer (e.g., computer 10) and in other embodiments thedata illustrated in memory 36 of computer 10 is hosted by severalcomputers (not shown). In fact, all possible arrangements of storing thedata illustrated in memory 36 of computer 10 on one or more computerscan be used so long as these components are addressable with respect toeach other across computer network 34 or by other electronic means.Thus, a broad array of computer systems can be used.

Projection Subsystem

The projection subsystem 270 obtains the constructed image from theimage constructor 256 (which optionally includes other information aboutthe subject, such as images obtained in one or more otherelectromagnetic bands), and projects the image onto the subject.Preferably, the image is projected such that representations of spectralfeatures are projected directly onto, or approximately onto, theconditions or physiological structures that generated those spectralfeatures. Examples of useful projection systems include liquid crystaldisplay (LCD) projectors and digital signal processing (DSP) projectors.

In some embodiments, some or all of the light forming the image thatprojection subsystem 270 projects is in a band that overlaps with lightin a band that the sensor subsystem 230 obtains. In order to inhibitunwanted feedback of the image light into the sensor subsystem 230, atimer can be used that synchronizes the projection subsystem 270 withthe sensor subsystem 230, so that the projection subsystem 270 does notproject an image while the sensor subsystem 230 is obtaining light fromthe subject. In some embodiments, the light forming the image thatprojection subsystem 270 projects does not spectrally overlap with thelight that the sensor subsystem 230 obtains.

As mentioned above, although projection of an image onto the subject canbe useful, the image can alternately be displayed on a video display,along with a normal (visible light) image of the subject. In suchembodiments the video display would replace the projection subsystem 270illustrated in FIG. 2. Or, the system could include both the videodisplay and the projection subsystem 270, allowing both modes of viewinginformation about the subject. The system can include a conventionalcamera for obtaining a normal (visible light) image of the area 210 ofthe subject, as well as other subsystems for obtaining different typesof information about the subject, which can optionally be included inthe projected and/or displayed image.

Medical Conditions

The systems and methods described herein can be used to determinewhether the subject has a wide variety of medical conditions. Someexamples include, but are not limited to: abrasion, alopecia, atrophy,av malformation, battle sign, bullae, burrow, basal cell carcinoma,burn, candidal diaper dermatitis, cat-scratch disease, contactdermatitis, cutaneous larva migrans, cutis marmorata, dermatoma,ecchymosis, ephelides, erythema infectiosum, erythema multiforme,eschar, excoriation, fifth disease, folliculitis, graft vs. hostdisease, guttate, guttate psoriasis, hand, foot and mouth disease,Henoch-Schonlein purpura, herpes simplex, hives, id reaction, impetigo,insect bite, juvenile rheumatoid arthritis, Kawasaki disease, keloids,keratosis pilaris, Koebner phenomenon, Langerhans cell histiocytosis,leukemia, lichen striatus, lichenification, livedo reticularis,lymphangitis, measles, meningococcemia, molluscum contagiosum,neurofibromatosis, nevus, poison ivy dermatitis, psoriasis, scabies,scarlet fever, scar, seborrheic dermatitis, serum sickness, Shagreenplaque, Stevens-Johnson syndrome, strawberry tongue, swimmers' itch,telangiectasia, tinea capitis, tinea corporis, tuberous sclerosis,urticaria, varicella, varicella zoster, wheal, xanthoma, zosteriform,basal cell carcinoma, squamous cell carcinoma, malignant melanoma,dermatofibrosarcoma protuberans, Merkel cell carcinoma, and Kaposi'ssarcoma.

Other examples include, but are not limited to: tissue viability (e.g.,whether tissue is dead or living, and/or whether it is predicted toremain living); tissue ischemia; malignant cells or tissues (e.g.,delineating malignant from benign tumors, dysplasias, precanceroustissue, metastasis); tissue infection and/or inflammation; and/or thepresence of pathogens (e.g., bacterial or viral counts). Someembodiments include differentiating different types of tissue from eachother, for example, differentiating bone from flesh, skin, and/orvasculature. Some embodiments exclude the characterization ofvasculature.

The levels of certain chemicals in the body, which may or may not benaturally occurring in the body, can also be characterized. For example,chemicals reflective of blood flow, including oxyhemoglobin anddeoxyhemoglobin, myoglobin, and deoxymyoglobin, cytochrome, pH, glucose,calcium, and any compounds that the subject may have ingested, such asillegal drugs, pharmaceutical compounds, or alcohol.

Other Embodiments

Some embodiments include a distance sensor (not shown) that facilitatespositioning the subject at an appropriate distance from the sensorand/or projector. For example, the system 200 can include a laser rangefinder that provides a visible and/or audible signal such as a lightand/or a beep or alarm, if the distance between the system and thesubject is not suitable for obtaining light from and/or projecting lightonto the subject. Alternately, the laser range finder may provide avisible and/or audible signal if the distance between the system and thesubject is suitable.

The illumination subsystem 210, sensor subsystem 230, processorsubsystem 250, and projection subsystem 270 can be co-located (e.g., allenclosed in a common housing).

Alternatively, a first subset of the subsystems can be co-located, whilea second subset of the subsystems are located separately from the firstsubset, but in operable communication with the first subset. Forexample, the illumination, sensing, and projection subsystems 210, 230,270 can be co-located within a common housing, and the processingsubsystem 250 located separately from that housing and in operablecommunication with the illumination, sensing, and projection subsystems.Alternatively, each of the subsystems can be located separately from theother subsystems. The storage 240 and storage 252 can be in the samedevice or in two separate devices, and that processor 238 of the sensorsubsystem may perform some or all of the functions of the spectralportion selector 245 and/or the image constructor 256 of the processorsubsystem 250.

Although illumination subsystem 210 is illustrated as irradiating anarea 201 that is of identical size to the area from which sensorsubsystem 230 obtains light and upon which projection subsystem 270projects the image, the areas need not be of identical size. Forexample, illumination subsystem 210 can irradiate an area that issubstantially larger than the region from which sensor subsystem 230obtains light and/or upon which projection subsystem 270 projects theimage. Also, the light from projection subsystem 270 may irradiate alarger area than sensor subsystem 230 senses, for example in order toprovide an additional area in which the subsystem 270 projects notationsand/or legends that facilitate the inspection of the projected image.Alternately, the light from projection subsystem 270 may irradiate asmaller area than sensor subsystem 230 senses.

Although illumination subsystem 210, sensor subsystem 230, andprojection subsystem 270 are illustrated as being laterally offset fromone another, resulting in the subject being irradiated with light comingfrom a different direction than the direction from which the sensorsubsystem 230 obtains light, and a different direction than thedirection from which the projection subsystem 270 projects the imageonto the subject. As will be apparent to those skilled in the art, thesystem can be arranged in a variety of different manners that will allowthe light to/from some or all of the components to be collinear, e.g.,through the use of dichroic mirrors, polarizers, and/or beamsplitters.Alternatively, multiple functionalities can be performed by a singledevice. For example, the projection subsystem 270 could also be used asthe irradiation subsystem 210, with timers used in order to irradiatethe subject and project the image onto the subject at slightly offsettimes.

In some embodiments, the spectral portion selector 254 has access tospectral information (e.g., characteristic wavelength bands and/ornormalized reflectances R_(N)(λ)) associated with a wide variety ofmedical conditions, physiological characteristics, and/or chemicals.This information can be stored, for example, in storage 252, or can beaccessed via the Internet (interface not shown). In some embodiments,the spectral portion selector has access to spectral information for anarrow subset of medical conditions, physiological features, orchemicals, that is, the system 200 is constructed to address only aparticular kind of condition, feature, or chemical.

Any of the methods disclosed herein can be implemented as a computerprogram product that includes a computer program mechanism embedded in acomputer-readable storage medium wherein the computer program mechanismcomprises computer executable instructions for performing suchembodiments. Any portion (e.g., one or more steps) of any of the methodsdisclosed herein can be implemented as a computer program product thatincludes a computer program mechanism embedded in a computer-readablestorage medium wherein the computer program mechanism comprises computerexecutable instructions for performing such portion of any such method.All or any portion of the steps of any of the methods disclosed hereincan be implemented using one or more suitably programmed computers orother forms of apparatus. Examples of apparatus include but are notlimited to the devices depicted in FIGS. 2 and 6.

Further still, any of the methods disclosed herein can be implemented inone or more computer program products. Some embodiments disclosed hereinprovide a computer program product that comprises executableinstructions for performing one or more steps of any or all of themethods disclosed herein. Such methods can be stored on a CD-ROM, DVD,ZIP drive, hard disk, flash memory card, USB key, magnetic disk storageproduct, or any other physical (tangible) computer readable media thatis conventional in the art. Such methods can also be embedded inpermanent storage, such as ROM, one or more programmable chips, or oneor more application specific integrated circuits (ASICs). Such permanentstorage can be localized in a server, 802.11 access point, 802.11wireless bridge/station, repeater, router, mobile phone, or otherelectronic devices.

Some embodiments provide a computer program product that contains any orall of the program modules shown in FIG. 6. These program modules can bestored on a CD-ROM, DVD, magnetic disk storage product, or any otherphysical computer-readable data or physical program storage product orany other physical (tangible) computer readable media that isconventional in the art. The program modules can also be embedded inpermanent storage, such as ROM, one or more programmable chips, or oneor more application specific integrated circuits (ASICs). Such permanentstorage can be localized in a server, 802.11 access point, 802.11wireless bridge/station, repeater, router, mobile phone, or otherelectronic devices.

All references cited herein are hereby incorporated by reference hereinin their entirety and for all purposes to the same extent as if eachindividual publication or patent or patent application was specificallyand individually indicated to be incorporated by reference in itsentirety for all purposes.

Many modifications and variations of this application can be madewithout departing from its spirit and scope, as will be apparent tothose skilled in the art. The specific embodiments described herein areoffered by way of example only, and the application is to be limitedonly by the terms of the appended claims, along with the full scope ofequivalents to which the claims are entitled.

1. A method of displaying medical information about a subject, thesubject having a plurality of regions of skin, the method comprising:resolving the light obtained from each region of the plurality ofregions of skin into a corresponding spectrum; selecting a portion ofeach spectrum, the selected portion including medical information aboutthe corresponding region; constructing, an image based on the selectedportion of each spectrum; and projecting the image onto the subject. 2.The method of claim 1, further comprising generating the light with alight source.
 3. The method of claim 1, wherein the light has at leastone of a broadband spectrum and a narrowband spectrum.
 4. The method ofclaim 1, wherein the light includes at least one of an ultravioletwavelength, visible wavelength, an infrared wavelength, and a terahertzwavelength.
 5. The method of claim 1, wherein resolving the lightobtained from each region of the plurality of regions of skin into acorresponding spectrum comprises passing the light into a spectrometer.6. The method of claim 5, wherein the spectrometer spatially separatesthe light into a plurality of component wavelengths, and records anintensity of each component wavelength of the plurality of componentwavelengths.
 7. The method of claim 1, wherein selecting the portion ofeach spectrum is based on at least one of: a spectral characteristic ofa predetermined medical condition, a spectral characteristic of apredetermined physiological feature, and a spectral characteristic of apredetermined chemical.
 8. The method claim 1, wherein selecting theportion of each spectrum comprises applying a digital filter to adigital signal representing the spectrum.
 9. The method of claim 1,wherein selecting the portion of each spectrum comprises applying atleast one of a band-pass filter and a band-block filter to the spectrum.10. The method of claim 1, wherein constructing the image comprisesassigning the selected portion of each spectrum to at least one of avisible color and an intensity.
 11. The method of claim 10, whereinprojecting the image onto the subject comprises projecting the at leastone of the visible color and the intensity onto the subject.
 12. Themethod of claim 1, further comprising: selecting a different portion ofeach spectrum, the different selected portion including differentmedical information about the corresponding region; and constructing anew image based on the different selected portion of each spectrum. 13.The method of claim 1, further comprising at least one of storing eachspectrum and storing the image.
 14. The method of claim 13, wherein eachspectrum is stored in a hyperspectral data cube.
 15. The method of claim14, wherein selecting a portion of each spectrum comprises at least oneof selecting a volume from the hyperspectral data cube and comparinginformation in the hyperspectral data cube to known spectral informationabout a medical condition.
 16. The method of claim 1, wherein thesubject is a human.
 17. The method of claim 1, wherein there is a delayof less than about one minute between resolving light obtained from eachregion of the plurality of regions of skin and projecting the image ontothe skin of the subject.
 18. A system for displaying medical informationabout a subject, the subject having a plurality of regions of skin, thesystem comprising: a spectrometer for resolving light obtained from eachregion of the plurality of regions of skin into a correspondingspectrum; logic for selecting a portion of each spectrum, the selectedportion including medical information about the corresponding region;logic for constructing an image based on the selected portion of eachspectrum; and a projector for projecting the image onto the subject. 19.The system of claim 18, further comprising a light source forirradiating the subject with the light.
 20. The system of claim 19,wherein the light source generates at least one of a broadband spectrumand a narrowband spectrum.
 21. The system of claim 18, wherein the lightincludes at least one of an ultraviolet wavelength, a visiblewavelength, an infrared wavelength, and a terahertz wavelength.
 22. Thesystem of claim 18, wherein the spectrometer comprises: a diffractiongrating for separating the light into a plurality of componentwavelengths; and a sensor for recording an intensity of each componentwavelength of the plurality of component wavelengths.
 23. The system ofclaim 18, wherein the logic selects the portion of each spectrum basedon at least one of: a spectral characteristic of a predetermined medicalcondition; a spectral characteristic of a predetermined physiologicalfeature; and a spectral characteristic of a predetermined chemical. 24.The system of claim 18, further comprising a digital filter fordigitally selecting the portion of a digital signal representing thespectrum.
 25. The system of claim 18, further comprising at least one ofa band-pass filter and a band-block filter for selecting the portion ofeach spectrum.
 26. The system of claim 18, wherein the logic forconstructing the image assigns the selected portion of each spectrum toat least one of a visible color and an intensity.
 27. The system ofclaim 26, wherein the projector projects the at least one of the visiblecolor and the intensity onto the skin of the subject.
 28. The system ofclaim 18, further comprising logic for: selecting a different portion ofeach spectrum, the different selected portion including differentmedical information about the corresponding region; and constructing anew image based on the different selected portion of each spectrum. 29.The system of claim 18, further comprising a storage medium for at leastone of storing the image and storing each spectrum.
 30. The system ofclaim 29, wherein the storage medium stores each spectrum in ahyperspectral data cube.
 31. The system of claim 30, wherein selecting aportion of each spectrum comprises at least one of selecting a volumefrom the hyperspectral data cube and comparing information in thehyperspectral data cube to known spectral information about a medicalcondition.
 32. The system of any claim 18, wherein the subject is ahuman.
 33. The system of claim 18, wherein there is a delay of less thanabout one minute between the spectrometer's resolution of light obtainedfrom each region of the plurality of regions of skin and the projector'sprojecting the image onto the subject.
 34. A computer-readable mediumstoring one or more computer programs executable by a computer fordisplaying medical information about a subject, the subject having aplurality of regions of skin, the one or more computer programscollectively encoding computer executable instructions for performingthe method comprising: obtaining a spectrum corresponding to each regionof the plurality of regions of skin; selecting a portion of eachspectrum, the selected portion including medical information about thecorresponding region; constructing an image based on the selectedportion of each spectrum; and providing the image to a projection devicefor projection onto the subject.
 35. The computer-readable medium ofclaim 34, wherein the method further comprises obtaining each spectrumfrom a spectrometer.
 36. The computer-readable medium of claim 34,wherein the selecting the portion of each spectrum is based on at leastone of: a spectral characteristic of a predetermined medical condition,a spectral characteristic of a predetermined physiological feature, anda spectral characteristic of a predetermined chemical.
 37. Thecomputer-readable medium of claim 34, wherein the selecting the portionof each spectrum comprises applying a digital filter to a digital signalrepresenting the spectrum.
 38. The computer-readable medium of claim 34,wherein the selecting the portion of each spectrum comprises applying atleast one of a band-pass filter and a band-block filter to the spectrum.39. The computer-readable medium of claim 34, wherein the method furthercomprises constructing the image by assigning the selected portion ofeach spectrum to at least one of a visible color and an intensity. 40.The computer-readable medium of claim 34, wherein the method furthercomprises: selecting a different portion of each spectrum, the differentselected portion including different medical information about thecorresponding region; and constructing a new image based on thedifferent selected portion of each spectrum.
 41. The computer-readablemedium of claim 34, wherein the method further comprises storing eachspectrum or storing the image.
 42. The computer-readable medium of claim34, wherein the method further comprises storing each spectrum in ahyperspectral data cube.
 43. The computer-readable medium of claim 42,wherein the method comprises instructions for selecting a portion ofeach spectrum by at least one of selecting a volume from thehyperspectral data cube and comparing information in the hyperspectraldata cube to known spectral information about a medical condition.
 44. Amethod of displaying medical information about a subject, the subjecthaving a plurality of regions of skin, the method comprising: resolvinglight obtained from each region of the plurality of regions of skin intoa corresponding spectrum; selecting a portion of each spectrum, theselected portion including medical information about the correspondingregion; constructing a spectral image based on the selected portion ofeach spectrum; combining the spectral image with other information aboutthe subject to form a composite image; and displaying the compositeimage.
 45. The method of claim 44, wherein displaying the compositeimage comprises projecting the composite image onto the subject.
 46. Themethod of claim 44, wherein displaying the composite image comprisesdisplaying the composite image on a video display.
 47. The method ofclaim 44, wherein selecting the portion of each spectrum is based on atleast one of: a spectral characteristic of a predetermined medicalcondition, a spectral characteristic of a predetermined physiologicalfeature, and a spectral characteristic of a predetermined chemical. 48.The method of claim 44, wherein the light includes at least one of anultraviolet wavelength, a visible wavelength, an infrared wavelength,and a terahertz wavelength.
 49. The method of claims 44, wherein theother information about the subject comprises an image of the subject.50. The method of claim 49, wherein the image of the subject is in atleast one of an ultraviolet band, a visible band, an infrared band, anda terahertz band.
 51. A system for displaying medical information abouta subject, the subject having a plurality of regions of skin, the systemcomprising: a spectrometer for resolving light obtained from each regionof the plurality of regions of skin into a corresponding spectrum; logicfor selecting a portion of each spectrum, the selected portion includingmedical information about the corresponding region; logic forconstructing a spectral image based on the selected portion of eachspectrum; logic for combining the spectral image with other informationabout the subject to form a composite image; and a display fordisplaying the composite image.
 52. The system of claim 51, wherein thedisplay comprises a projector for projecting the composite image ontothe subject.
 53. The system of claim 51, wherein the display comprises avideo display.
 54. The system of claim 51, wherein the logic selects theportion of each spectrum based on at least one of: a spectralcharacteristic of a predetermined medical condition, a spectralcharacteristic of a predetermined physiological feature, and a spectralcharacteristic of a predetermined chemical.
 55. The system of claim 51,further comprising a light source for irradiating the subject with thelight.
 56. The system of claim 55, wherein the light includes at leastone of an ultraviolet wavelength, a visible wavelength, an infraredwavelength, and a terahertz wavelength.
 57. The system of claims 51,further comprising an imager for obtaining an image of the subject, andwherein the other information about the subject comprises the image ofthe subject.
 58. The system of claim 57, wherein the image of thesubject is in at least one of an ultraviolet band, a visible band, aninfrared band, and a terahertz band.
 59. A computer-readable mediumstoring one or more computer programs executable by a computer fordisplaying medical information about a subject, the subject having aplurality of regions of skin, the one or more computer programscollectively encoding computer executable instructions for performingthe method comprising: resolving light obtained from each region of theplurality of regions of skin into a corresponding spectrum; selecting aportion of each spectrum, the selected portion including medicalinformation about the corresponding region; constructing a spectralimage based on the selected portion of each spectrum; combining thespectral image with other information about the subject to form acomposite image; and displaying the composite image.
 60. Thecomputer-readable medium of claim 59, wherein the method furthercomprises obtaining each spectrum from a spectrometer.
 61. Thecomputer-readable medium of claim 59, wherein the displaying thecomposite image comprises projecting the composite image onto thesubject.
 62. The computer-readable medium of claim 59, wherein thedisplaying the composite image comprises displaying the composite imageon a video display.
 63. The computer-readable medium of claim 59,wherein the selecting the portion of each spectrum comprises selectingthe portion of each spectrum based on at least one of: a spectralcharacteristic of a predetermined medical condition, a spectralcharacteristic of a predetermined physiological feature, and a spectralcharacteristic of a predetermined chemical.
 64. The computer-readablemedium of claim 59, wherein the other information about the subjectcomprises an image of the subject at one of an ultraviolet wavelength, avisible wavelength, an infrared wavelength, and a terahertz wavelength.65. The method of claim 1, wherein there is a delay of less than aboutten seconds between resolving light obtained from each region of theplurality of regions of skin and projecting the image onto the subject.66. The method of claim 1, wherein there is a delay of less than aboutfive minutes between resolving light obtained from each region of theplurality of regions of skin and projecting the image onto the subject.67. The method of claim 1, wherein any combination of the resolving,selecting, constructing, and projecting is performed using a suitablyprogrammed computer.
 68. The method of claim 1, wherein the resolving isperformed using a suitably programmed computer.
 69. The method of claim1, wherein the selecting is performed using a suitably programmedcomputer.
 70. The method of claim 1, wherein the constructing isperformed using a suitably programmed computer.
 71. The method of claim1, wherein the projecting is performed using a suitably programmedcomputer.
 72. The system of claim 18, wherein there is a delay of lessthan about ten seconds between the spectrometer's resolution of lightobtained from each region of the plurality of regions of skin and theprojector's projecting the image onto the subject.
 73. The system ofclaim 18, wherein there is a delay of less than about five minutesbetween the spectrometer's resolution of light obtained from each regionof the plurality of regions of skin and the projector's projecting theimage onto the subject.
 74. The method of claim 44 wherein anycombination of the resolving, selecting, constructing, combining, anddisplaying is performed using a suitably programmed computer.
 75. Themethod of claim 44, wherein the resolving is performed using a suitablyprogrammed computer.
 76. The method of claim 44, wherein the selectingis performed using a suitably programmed computer.
 77. The method ofclaim 44, wherein the constructing is performed using a suitablyprogrammed computer.
 78. The method of claim 44, wherein the combiningis performed using a suitably programmed computer.
 79. The method ofclaim 44, wherein the displaying is performed using a suitablyprogrammed computer.